Insights into cosmological structure formation with machine learning

Our modern understanding of cosmological structure formation posits that small matter density fluctuations present in the early Universe, as traced by the cosmic microwave background, grow via gravitational instability to form extended haloes of dark matter. A theoretical understanding of the structure, evolution and formation of dark matter haloes is an essential step towards unravelling the intricate connection between halo and galaxy formation, needed to test our cosmological model against data from upcoming galaxy surveys. Physical understanding of the process of dark matter halo formation is made difficult by the highly non-linear nature of the haloes' evolution. I describe a new approach to gain physical insight into cosmological structure formation based on machine learning. This approach combines the ability of machine learning algorithms to learn non-linear relationships, with techniques that enable us to physically interpret the learnt mapping. I describe applications of the method, with the aim of investigating which aspects of the early universe density field impact the later formation of dark matter haloes. First I present a case where the process of halo formation is turned into a binary classification problem; the algorithm predicts whether or not dark matter `particles' in the initial conditions of a simulation will collapse into haloes of a given mass range. Second, I present its generalization to regression, where the algorithm infers the final mass of the halo to which each particle will later belong. I show that the initial tidal shear does not play a significant role compared to the initial density field in establishing final halo masses. Finally, I demonstrate that extending the framework to deep learning algorithms such as convolutional neural networks allows us to explore connections between the early universe and late time haloes beyond those studied by existing analytic approximations of halo collapse

Machine learning cosmological structure formation

We train a machine learning algorithm to learn cosmological structure formation from N-body simulations. The algorithm infers the relationship between the initial conditions and the final dark matter haloes, without the need to introduce approximate halo collapse models. We gain insights into the physics driving halo formation by evaluating the predictive performance of the algorithm when provided with different types of information about the local environment around dark matter particles. The algorithm learns to predict whether or not dark matter particles will end up in haloes of a given mass range, based on spherical overdensities. We show that the resulting predictions match those of spherical collapse approximations such as extended Press-Schechter theory. Additional information on the shape of the local gravitational potential is not able to improve halo collapse predictions; the linear density field contains sufficient information for the algorithm to also reproduce ellipsoidal collapse predictions based on the Sheth-Tormen model. We investigate the algorithm's performance in terms of halo mass and radial position and perform blind analyses on independent initial conditions realisations to demonstrate the generality of our results.Comment: 10 pages, 7 figures. Minor changes to match version published in MNRAS. Accepted on 22/06/201

Halo assembly bias from a deep learning model of halo formation

We build a deep learning framework that connects the local formation process of dark matter halos to the halo bias. We train a convolutional neural network (CNN) to predict the final mass and concentration of dark matter halos from the initial conditions. The CNN is then used as a surrogate model to derive the response of the halos' mass and concentration to long-wavelength perturbations in the initial conditions, and consequently the halo bias parameters following the "response bias" definition. The CNN correctly predicts how the local properties of dark matter halos respond to changes in the large-scale environment, despite no explicit knowledge of halo bias being provided during training. We show that the CNN recovers the known trends for the linear and second-order density bias parameters $b_1$ and $b_2$, as well as for the local primordial non-Gaussianity linear bias parameter $b_\phi$. The expected secondary assembly bias dependence on halo concentration is also recovered by the CNN: at fixed mass, halo concentration has only a mild impact on $b_1$, but a strong impact on $b_\phi$. Our framework opens a new window for discovering which physical aspects of the halo's Lagrangian patch determine assembly bias, which in turn can inform physical models of halo formation and bias.Comment: 11 pages, 5 figures, to be submitted to MNRAS, comments welcom

The causal effect of environment on halo mass and concentration

Understanding the impact of environment on the formation and evolution of dark matter halos and galaxies is a crucial open problem. Studying statistical correlations in large simulated populations sheds some light on these impacts, but the causal effect of an environment on individual objects is harder to pinpoint. Addressing this, we present a new method for resimulating a single dark matter halo in multiple large-scale environments. In the initial conditions, we 'splice' (i.e. insert) the Lagrangian region of a halo into different Gaussian random fields, while enforcing consistency with the statistical properties of $\Lambda$CDM. Applying this technique, we demonstrate that the mass of halos is primarily determined by the density structure inside their Lagrangian patches, while the halos' concentration is more strongly affected by environment. The splicing approach will also allow us to study, for example, the impact of the cosmic web on accretion processes and galaxy quenching.Comment: 6 pages, 5 figures. Accepted 2021 September 10. Received 2021 September 9; in original form 2021 July

A robust estimator of mutual information for deep learning interpretability

We develop the use of mutual information (MI), a well-established metric in information theory, to interpret the inner workings of deep learning (DL) models. To accurately estimate MI from a finite number of samples, we present GMM-MI (pronounced ‘Jimmie’), an algorithm based on Gaussian mixture models that can be applied to both discrete and continuous settings. GMM-MI is computationally efficient, robust to the choice of hyperparameters and provides the uncertainty on the MI estimate due to the finite sample size. We extensively validate GMM-MI on toy data for which the ground truth MI is known, comparing its performance against established MI estimators. We then demonstrate the use of our MI estimator in the context of representation learning, working with synthetic data and physical datasets describing highly non-linear processes. We train DL models to encode high-dimensional data within a meaningful compressed (latent) representation, and use GMM-MI to quantify both the level of disentanglement between the latent variables, and their association with relevant physical quantities, thus unlocking the interpretability of the latent representation. We make GMM-MI publicly available in this GitHub repository

A robust estimator of mutual information for deep learning interpretability

We develop the use of mutual information (MI), a well-established metric in information theory, to interpret the inner workings of deep learning models. To accurately estimate MI from a finite number of samples, we present GMM-MI (pronounced $``$Jimmie$"$), an algorithm based on Gaussian mixture models that can be applied to both discrete and continuous settings. GMM-MI is computationally efficient, robust to the choice of hyperparameters and provides the uncertainty on the MI estimate due to the finite sample size. We extensively validate GMM-MI on toy data for which the ground truth MI is known, comparing its performance against established mutual information estimators. We then demonstrate the use of our MI estimator in the context of representation learning, working with synthetic data and physical datasets describing highly non-linear processes. We train deep learning models to encode high-dimensional data within a meaningful compressed (latent) representation, and use GMM-MI to quantify both the level of disentanglement between the latent variables, and their association with relevant physical quantities, thus unlocking the interpretability of the latent representation. We make GMM-MI publicly available.Comment: 30 pages, 8 figures. Minor changes to match version accepted for publication in Machine Learning: Science and Technology. GMM-MI available at https://github.com/dpiras/GMM-M

Cancer risk and tumour spectrum in 172 patients with a germline SUFU pathogenic variation : a collaborative study of the SIOPE Host Genome Working Group

Background Little is known about risks associated with germline SUFU pathogenic variants (PVs) known as a cancer predisposition syndrome. Methods To study tumour risks, we have analysed data of a large cohort of 45 unpublished patients with a germline SUFU PV completed with 127 previously published patients. To reduce the ascertainment bias due to index patient selection, the risk of tumours was evaluated in relatives with SUFU PV (89 patients) using the Nelson-Aalen estimator. Results Overall, 117/172 (68%) SUFU PV carriers developed at least one tumour: medulloblastoma (MB) (86 patients), basal cell carcinoma (BCC) (25 patients), meningioma (20 patients) and gonadal tumours (11 patients). Thirty-three of them (28%) had multiple tumours. Median age at diagnosis of MB, gonadal tumour, first BCC and first meningioma were 1.5, 14, 40 and 44 years, respectively. Follow-up data were available for 160 patients (137 remained alive and 23 died). The cumulative incidence of tumours in relatives was 14.4% (95% CI 6.8 to 21.4), 18.2% (95% CI 9.7 to 25.9) and 44.1% (95% CI 29.7 to 55.5) at the age of 5, 20 and 50 years, respectively. The cumulative risk of an MB, gonadal tumour, BCC and meningioma at age 50 years was: 13.3% (95% CI 6 to 20.1), 4.6% (95% CI 0 to 9.7), 28.5% (95% CI 13.4 to 40.9) and 5.2% (95% CI 0 to 12), respectively. Sixty-four different PVs were reported across the entire SUFU gene and inherited in 73% of cases in which inheritance could be evaluated. Conclusion Germline SUFU PV carriers have a life-long increased risk of tumours with a spectrum dominated by MB before the age of 5, gonadal tumours during adolescence and BCC and meningioma in adulthood, justifying fine-tuned surveillance programmes.Peer reviewe